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DETAILED ACTION 
Claim Objections 

1 . Claim 1 1 is objected to because of tlie following infomnalities 

• Claim 1 1 , line 9: "extracted one" sliould have been "extracted by one" 

Appropriate correction is required. 

Claim Rejections - 35 USC §112 

2. The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

3. Claims 1-4 and 6-12 are rejected under 35 U.S.C. 112, second paragraph, as 
being indefinite for failing to particularly point out and distinctly claim the subject matter 
which applicant regards as the invention. 

4. Claim 1 recites the limitation "the person" in line 12. Since it is not clear whether 
it is the person in line 3 or line 10 that is being referred to, the mete and bound of the 
claim cannot be ascertained. Claims 2 and 3 are similarly rejected due to dependency. 
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5. Claim 2 recites tlie limitation "the image" in line 2. Since a camera can pick up 
more than one Image, it is not clear which one is being referred to and therefore the 
mete and bound of the claim cannot be ascertained. Claim 3 is similarly rejected due to 
dependency. 

6. Claim 3 recites the limitation "the other" in line 5 and claims 4 and 6 recite the 
same In their respective line 7. Since a plurality can be more than 2, it Is not clear 
which one is being referred to and therefore the mete and bound of the claim cannot be 
ascertained. 

7. Claim 4, and similarly claims 7 and 12, recites the limitation "the person" In the 
3^*^ and 4"^ lines from the end. Since there are two sets of face feature each implicitly 
corresponding to a (possibly different) face, it is not clear which one is being referred to 
and therefore the mete and bound of the claim cannot be ascertained. 

8. Claim 6 recites "to convert one of the plurality of face feature points to the other 
of the plurality of face feature points" in lines 6-8, The meaning of converting one 
feature point to another is not clear and therefore the mete and bound of the claim 
cannot be ascertained. [Note: Fig. 7, refs. S4-S6; P. 26, lines 10-25 and P. 27. line 10- 
P. 28, line 4 describe establishing corresponding relationship among feature points to 
determine its 3-D coordinates and then projects the 3-D coordinates into the coordinate 
system of a camera to which the feature point cannot be extracted (e.g., because it's 
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obscured and therefore not visible to the camera); in other words, predicts where the 
feature point would have been in the image captured by that camera. However, since 
the predicted feature point is cannot be extracted, it is not among "the plurality of face 
feature points;" therefore converting one of the plurality of face feature points to "the 
other of the plurality efface feature points" does not make sense.] 

9. Claim 7 recites the limitation "the face region" in line 9. Since there are more 
than one face regions (per lines 4-6), it is therefore not clear which one is being referred 
to, and the mete and bound of the claim cannot be ascertained. Claims 8-10 are 
similarly rejected due to dependency. 

10. Claim 7 further recites the limitation "a face feature from images" in lines 8 and 9. 
Since it can be construed either as "one feature from all the images combined" or "one 
feature per image," ambiguity arises and the mete and bound of the claim cannot be 
ascertained. Claims 8-10 are similarly rejected due to dependency. 

1 1 . Claim 8 recites the limitation "a plurality of the face features" in lines 5 & 6. 
There is insufficient antecedent basis for this limitation in the claim (since claim 7 only 
recite "a face feature" in lines 8 and 9). Claim 9 is similarly rejected due to dependency. 
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1 2. Claims 9-1 1 recite tlie limitation "tlie other" in line 4, line 4 and line 10, 
respectively. Since a plurality can be more than 2. it is not clear which one is being 
referred to and therefore the mete and bound of the claim cannot be ascertained. 

13. Claim 10 recites the limitation "the corresponding face feature point" in the last 
two lines. Since more than one face feature point have been recited ("one of in line 3 
and "the other" in line 4), it is not clear which one is being referred to and therefore the 
mete and bound of the claim cannot be ascertained. 

14. Claim 1 1 recites the limitation "the face feature point" in the 2"" line from the end. 
Since more than one face feature point have been recited (in lines 8 and 10, 
respectively), it is not clear which one is being referred to and therefore the mete and 
bound of the claim cannot be ascertained. 

Claim Rejections - 35 USC § 103 

1 5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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16. Claims 1, 4, 6. 12 and 14 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Prokoski (US 6,920,236) and in view of Li (US 7,1 55,036). 



17. Regarding claim 1, and similarly claim 12, Prokoski discloses an apparatus that 
uses multiple imaging devices [Fig. 12A, refs. 102 & 202 and Fig. 13, ref. 302 (imaging 
units)] that comprises 

• a face feature extracting unit configured to extract a face feature 
from an image of the face region detected by the face region detecting 
unit 

[Fig. 13, refs. 304 & 404 (collectively the face feature extracting 
unit); Col. 13, lines 45-50. Note that refs. 304 and 404 each 
generates a morph facial image, which is a face feature per se (see P. 
12, lines 10-12 of the instance application)] 

• a person recognizing unit configured to calculate a similarity measure 
based on a face feature of a specific person being previously 
registered and the face feature extracted by the face feature 
extracting unit to recognize the person 

[Fig. 13, ref. 324; Col. 14, lines 23-25] 

• an output apparatus which outputs a recognition result in the person 
recognizing unit 

[Fig. 13, ref. 325; Col. 14, lines 25-26] 



Prokoski does not expressly disclose the following, which is taught by Li 

• a face region detecting unit configured to detect a face feature point 
of a person from a plurality of images picked up by a^ plurality of 
imaging units respectively, to detect a face region 

[Fig. 1, refs. 120 & 124 (feature points) and 134-140 (detect face); 
Col. 4, line 36-Col. 6, line 6. Note that Prolcoski already discloses 
multiple images from multiple imaging units] 



Prokoski and Li are combinable because they both have aspects that are from the same 
field of endeavor of face detection. 
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At the time tlie invention, it would liave been obvious to one of ordinary sl^ill in tlie art to 
modify Prokoski as recited above. The motivation would have been to be able to 
augment the capability of Prokoski's invention by providing the ability to extract faces 
from images when other parts of a human body is also captured by the imaging units 
(as is common in a surveillance application), as well as to overcome problems 
encountered in traditional face detection techniques, as Li indicates in Col. 1 , line 34- 
Col. 2, line 2. 

Therefore, it would have been obvious to combine Li with Prokoski to obtain the 
invention as specified in claim 1 . 

1 8. Regarding claim 4, note that per the analysis of claim 1 above, Li discloses 
detecting multiple feature points [Fig. 1, refs. 120 and 124]. In addition, Prokoski further 
discloses predicting a feature point from another [Fig. 12A, refs. 148, 150 & 254; Col. 
12, lines 47-59; note that the eye line is considered the predicted feature point] and 
detecting at least two face feature regions [Fig. 12A, refs. 110 (from IR camera) & 210 
(from visual camera); Fig. 13, refs.304 & 404]. Since Li relies on feature points to detect 
face regions, it would have been obvious to one of ordinary skill in the art to modify the 
combined invention of Prokoski and Li with Prokoski's disclosure above to predict (the 
missing) feature points that are needed by Li in order to successfully detect face 
regions. (Note that face feature extraction, person recognition and output units have all 
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been disclosed in the combined invention of Prol^oski and Li, per the analysis of claim 1 
above.) 

19. Regarding claim 6, and similarly claim 14, note that the combined invention of 
Prokoski and Li discloses the recited detecting unit (per the analysis of claim 1 ). Note 
further that using stereo matching (or other similar triangulation techniques) to 
detemiine 3-D coordinates of a feature point as appeared on multiple Images (e.g. a 
stereo pair) is well known in the art. In addition, projecting a 3-D point onto to a 
camera's image plane as seen from a given viewpoint to predict its image coordinates is 
also well known. Therefore it would have been obvious to apply the techniques in order 
to estimate where a face feature point would have been in the image captured by a 
camera to which that feature point is not visible. [See, for example, JP 2000-163469 
and JP 59-182689 (as citations 2 and 3 in the English translation of JPO Office action 
filed 04/19/07) and the discussion on applying these patents to claim rejections in the 
JPO Office action. Note that both the above-cited patents and Office action are part of 
the IDS.] 

***** 

20. Claims 2 and 7 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Prokoski (US 6,920,236) and Li (US 7,155,036) as applied to claims 1 and 12 above, 
and further in view of Howard et al. (US 2004/0213437). 
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Regarding claim 2, and similarly claim 7, the combined invention of Prokoski and Li 
discloses all limitations of its parent, claim 1 but not the following 

• wherein the face region detecting unit includes a plurality of face 
region detecting sections each configured to detect the face feature 
point of the person from the image picked up by one of the plurality of 
imaging units to detect the face region 

• the face feature detecting unit includes a plurality of face feature 
extracting sections each configured to extract the face feature from 
the image of the face region detected by one of the plurality of face 
detecting sections 

However, Howard discloses using multiple units to perform the same task such as face 
recognition and eye (a face feature) detection [Fig. 16, refs. 1218-1222 and Col. 14, 
lines 23-25]. 

The combined invention of Prokoski and Li is combinable with Howard because they 
both have aspects that are from the same field of endeavor of face recognition. 

At the time the invention, it would have been obvious to one of ordinary skill in the art to 
modify the combined invention of Prokoski and Li with Howard by using multiple 
sections for the face region detecting unit, as well as for the face feature extracting unit. 
The motivation would have been to improve the processing speed by doing face region 
detection and feature extraction from multiple images in parallel, as would have been 
obvious to one of ordinary skill in the art. 

Therefore, it would have been obvious to combine Howard with Prokoski and Li to 
obtain the invention as specified in claims 2 and 7. 
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***** 

21. Claims 5 and 13 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Prokoski (US 6.920,236) and Li (US 7,155,036), and further in view of Nakamura (US 
7,212,233). 

Regarding claim 5, and similarly claim 13, per the analysis of claim 1 above the 
combined invention of Prokoski and Li discloses the recited face region detection unit 
but not the following: 

• a predicting unit configured to compare a feature of the detected face 
feature region with respective features of a face feature region of a 
person facing towards a plurality of predetermined directions to detect 
a face direction of the face 

However, Nakamura discloses a unit that compares a feature of a face region with 
respective features (as encoded in detection network) to detect a face direction [Fig. 2, 
ref. 102 (prediction unit); Fig. 4, refs. 204-204B (features of faces facing different 
directions) and 206A-206C (detection networks); Fig. 5, refs. S14-S18; Fig. 7 (learning 
to build the detection network) and Fig. 9 (comparing); Col. 6, line 38-Col. 8, line 46. 
Note that the feature used is created in S22 of Fig. 7 and S33 of Fig. 9, respectively]. 

The combined invention of Prokoski and Li is combinable with Nakamura because they 
both have aspects that are from the same field of endeavor of face recognition: 
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At the time the invention, it would have been obvious to one of ordinary skill in the art to 
modify the combined invention of Prokoski and Li with Nakamura as recited above. The 
motivation would have been to provide the ability to obtain images when the subject is 
in a desired pose, as Nakamura indicates in Col. 1 , lines 53-56. 

Therefore, it would have been obvious to combine Nakamura with Prokoski and Li to 
obtain the invention as specified in claim 5. 

***** 

22. Claim 10 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Prokoski (US 6,920,236), Li (US 7,155,036) and Howard et al. (US 2004/0213437) as 
applied to claims 2 and 7 above, and further in view of Nakamura (US 7,212,233). 

Regarding claim 10, the combined invention of Prokoski, Li and Howard discloses all 
limitations of its parent, claim 7. 

The combined invention does not expressly discloses the following: 

• a predicting unit configured to compare a feature of the detected face 
feature region with respective features of a face feature region of a 
person facing towards a plurality of predetermined directions to detect 
a face direction of the face 

However, Nakamura discloses a unit that compares a feature of a face region with 
respective features (as encoded in detection network) to detect a face direction [Fig. 2, 
ref. 102 (prediction unit); Fig. 4, refs. 204-204B (features effaces facing different 
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directions) and 206A-206C (detection networl<s); Fig. 5, refs. S14-S18; Fig. 7 (learning 
to build the detection network) and Fig. 9 (comparing); Col. 6, line 38-Col. 8, line 46. 
Note that the feature used Is created In S22 of Fig. 7 and S33 of Fig. 9, respectively]. 

The combined Invention of Prokosl<i. LI and Howard is comblnable with Nakamura 
because they both have aspects that are from the same field of endeavor of face 
recognition. 

At the time the Invention, It would have been obvious to one of ordinary skill in the art to 
modify the combined Invention of Prokoski, LI and Howard with Nakamura as recited 
above. The motivation would have been to provide the ability to obtain Images when 
the subject is In a desired pose, as Nakamura indicates In Col. 1, lines 53-56. 

Therefore, it would have been obvious to combine Nakamura with Prokoski, LI and 
Howard to obtain the Invention as specified In claim 10. 

Allowable Subject Matter 

23. The following is an examiner's statement of the Indication of allowable subject 
matter: 
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A. Regarding claim 3, and similarly claim 8, prior art of record, alone or in 
combination, does not disclose, teach or suggest comparing a face feature 
extracted from different images to determine whether the subject being imaged is 
a photograph. Prokoski discloses using IR images to determine whether the 
subject is a live face or not, but not whether it is a photograph. In addition, Fan 
et al. ("Picture/Graphics Classification Using Texture Features," SPIE, Vol. 4663, 
2002, PP. 81-85) discloses determining whether an image corresponds to a 
natural object or synthetic graphics; however, it does not compare features from 
images obtained by different cameras. Claim 9 inherits this subject matter from 
claim 8 

B. Regarding claim 1 1 , prior art of record, alone or in combination, does not 
disclose, teach or suggest recognizing a person only after determining that the same 
person is indicated by the feature points extracted from two different images. Kaku (US 
7,035,440) discloses identifying the same person from images captured by multiple 
cameras [Fig. 22, ref. 308], but does not using it as a criterion as to whether to 
recognize the person or not. 

Conclusion and Contact Information 

24. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure: 
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• Fukui et al. (US 6,466,685) - discloses detecting face regions using template 
matching or color and subsequent extraction of features for recognition [Fig. 4; 
Col. 7, line66-Col. 8, line 11] 

• Engels et al. (US 7,215,798) - discloses detemnining whether a captured 
fingerprint image is from a live finger or. not [Fig. 2B] 

25. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Yubin Hung whose telephone number Is (571 ) 272- 
7451 . The examiner can nomially be reached on 7:30 - 4:00. If attempts to reach the 
examiner by telephone are unsuccessful, the examiner's supervisor, Matthew C. Bella 
can be reached on (571) 272-7778. The fax phone number for the organization where 
this application or proceeding is assigned is 571-273-8300. 
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26. Information regarding the status of an application may be obtained from the 
Patent Application Infonnation Retrieval (PAIR) system. Status infomiation for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more infomiation about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 




Yubin Hung 
Patent Examiner 
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May 29, 2007 



